Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADDED] ListKeys method for listing kv keys #1490

Merged
merged 1 commit into from
Dec 13, 2023
Merged

[ADDED] ListKeys method for listing kv keys #1490

merged 1 commit into from
Dec 13, 2023

Conversation

piotrpio
Copy link
Collaborator

Signed-off-by: Piotr Piotrowski piotr@synadia.com

Signed-off-by: Piotr Piotrowski <piotr@synadia.com>
@piotrpio piotrpio requested a review from Jarema December 12, 2023 14:20
@@ -67,7 +67,10 @@ type (
// WatchAll will invoke the callback for all updates.
WatchAll(ctx context.Context, opts ...WatchOpt) (KeyWatcher, error)
// Keys will return all keys.
// DEPRECATED: Use ListKeys instead to avoid memory issues.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which was the memory issue?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keys() could be used to fetch millions of keys all stored in a slice before returning. There should be a way of either paging or iterating over the result (which the new method provides). I could add some details to this comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a use case now with > 600k keys.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah I see, makes sense to switch to use channels instead

Copy link
Member

@wallyqs wallyqs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@Jarema Jarema left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Will add note to ADR that such scalable method should be available in all clients, if any without iterators had similar approach.

@piotrpio piotrpio merged commit b95565d into main Dec 13, 2023
2 checks passed
@piotrpio piotrpio deleted the kv-keys-lister branch December 13, 2023 16:03
@piotrpio piotrpio mentioned this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants